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UAVs  and  control  delays 


"Unmanned  Aerial  Vehicles  worden  op  afstand  aangestuurd  via  een 
verbinding  die  een  flinke  vertraging  kan  introduceren.  Hoe  groot  kan  die 
vertraging  worden  en  wat  is  eigenlijk  nog  acceptabel?" 


Probleemstelling 

UAV’s  (Unmanned  Aerial  Vehicles) 
worden  vanuit  een  GCS  (Ground  Control 
Station)  op  afstand  aangestuurd.  Vanwege 
de  aard  van  de  verbinding  van  deze  twee 
eenheden  is  er  alti  jd  sprake  van  een  zekere 
mate  van  vertraging  van  de  stuursignalen 
van  het  GCS  naar  de  UAV  en  van  de 
feedback  (telemetrie  en  sensorinformatie) 
van  de  UAV  naar  het  GCS.  Deze 
vertragingen  kunnen  een  invloed  hebben  op 
de  kwaliteit  van  de  aansturing  van  de  UAV. 


De  Koninklijke  Luchtmacht  bezint  zich  op 
een  mogelijke  aanschaf  van  UAV’s.  Het  is 
daarbij  van  belang  te  weten  welke 
beperkingen  de  verschillende  verbindingen 
opleveren  en  welke  mogelijke  oplossingen 
mogelijk  zouden  zijn.  TNO  voerde  binnen 
het  programma  Nieuwe  Generatie 
Gevechtsvliegtuigen  een  onderzoek  uit 
specifiek  gericht  op  het  gebied  van 
signaalvertragingen.  Overige  aspecten  van 
de  verbinding  (bijvoorbeeld  bandbreedte, 
continui'teit  van  de  verbinding,  ruis  en 
betrouwbaarheid)  maakten  geen  deel  uit  van 
de  onderzoeksopdracht 


Beschrijving  van  de 
werkzaamheden 

Onze  werkzaamheden  bestonden  uit  het 
verwerken  van  literatuur  met  betrekking  tot 
mogelijke  bronnen  van  vertraging  en 
rekenwerk  op  dit  gebied,  het  verwerken  van 
literatuur  op  het  gebied  van  effecten  van 
vertraging  op  de  menselijke  prestatie  en 
literatuur  op  het  gebied  van  mogelijke 
oplossingen  voor  de  problematiek  die  de 
vertragingen  opleveren. 

Resultaten  en  conclusies 

Het  gevonden  bereik  van  vertragingen  voor 
de  verschillende  operatiewijzen  van  UAVs 
is  zeer  groot.  De  bijbehorende  effecten  op 
de  prestatie  van  de  bestuurder  varieren  van 
net  meetbaar  tot  onacceptabel.  Er  zijn 
echter  diverse  compensatiemethoden  die  het 
bereik  van  de  acceptabele  vertragingen  tlink 
kunnen  oprekken. 

Toepasbaarheid 

De  resultaten  zijn  bruikbaar  in  het  UAV- 
verwervingsproces.  Het  is  voorstelbaar  dat 
bijvoorbeeld  een  keuze  gemaakt  moet 
worden  uit  een  systeem  dat  direct  moet 
worden  aangestuurd  (als  een  vliegtuig)  of 
een  systeem  dat  meer  op  basis  van 
supervisie  wordt  aangestuurd.  De  resultaten 
van  de  studie  kunnen,  samen  met  gegevens 
over  de  door  de  KLu  gewenste 
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verbindingswijze,  aangeven  of  het  platform 
nog  acceptabel  aan  te  sturen  is  en  daarmee 
dus  een  valide  kandidaat  is. 
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Samenvatting 


Vraagstelling: 

UAVs  (Unmanned  Aerial  Vehicles)  worden  vanuit  een  GCS  (Ground  Control  Station) 
op  afstand  aangestuurd.  Vanwege  de  aard  van  de  verbinding  van  deze  twee  eenheden  is 
er  altijd  sprake  van  een  zekere  mate  van  vertraging  van  de  stuursignalen  van  het  GCS 
naar  de  UAV  en  van  de  feedback  (telemetrie  en  sensorinformatie)  van  de  UAV  naar  het 
GCS.  Deze  vertragingen  kunnen  een  invloed  hebben  op  de  kwaliteit  van  de  aansturing 
van  de  UAV. 

De  Koninklijke  Luchtmacht  bezint  zich  op  een  mogelijke  aanschaf  van  UAV’s.  Het  is 
daarbij  van  belang  te  weten  welke  beperkingen  de  verschillende  verbindingen  opleveren 
en  welke  mogelijke  oplossingen  mogelijk  zouden  zijn.  TNO  voerde  binnen  het 
programma  Nieuwe  Generatie  Gevechtvliegtuigen  een  onderzoek  uit  specifiek  gericht 
op  het  gebied  van  signaalvertragingen.  Overige  aspecten  van  de  verbinding 
(bijvoorbeeld  bandbreedte,  continuiteit  van  de  verbinding,  ruis  en  betrouwbaarheid) 
maakten  geen  deel  uit  van  de  onderzoeksopdracht. 

Werkwijze: 

Onze  werkzaamheden  bestonden  uit  het  verwerken  van  literatuur  op  het  gebied  van  de 
mogelijke  bronnen  van  vertraging  en  enig  rekenwerk  op  dit  gebied,  het  verwerken  van 
literatuur  op  het  gebied  van  effecten  van  vertraging  op  de  menselijke  prestatie  en 
literatuur  op  het  gebied  van  mogelijke  oplossingen  voor  de  problematiek  die  de 
vertragingen  opleveren. 

Resultaten: 

Afhankelijk  van  de  specifieke  situatie  blijken  vertragingen  tussen  de  100  en  1600  ms 
(en  meer)  op  te  kunnen  treden.  Dit  is  aanzienlijk,  aangezien  er  bij  100  ms  al  meetbare 
verslechtering  van  de  menselijke  stuurprestatie  optreedt,  en  er  bij  een  vertraging  van 
250-300  ms  in  het  geval  van  vliegtuigbesturing  al  meestal  van  een  onacceptabele 
verlaging  van  de  vliegkwaliteiten  sprake  is.  Er  blijken  echter  technieken  te  zijn 
(filtering  en  voorspellende  displays)  die  het  traject  van  de  acceptabele  vertraging 
kunnen  oprekken  tot  zo’n  400  ms  of  meer.  Dit  laatste  geldt  voor  de  handmatige 
besturing  van  de  UAV  zelf.  De  handmatige  besturing  van  camera’s  aan  boord  van 
UAV’s  is  aanzienlijk  minder  kritisch  en  zal  waarschijnlijk  tot  ruim  voorbij  de  1000  ms 
via  genoemde  aanpassingen  tot  een  acceptabel  resultaat  kunnen  leiden. 

Conclusies: 

Bij  directe  verbindingen  tussen  GCS  en  UAV’s,  dan  wel  bij  verbindingen  met 
relayering  via  tussenstations  op  het  land  of  laagvliegende  relaisstations,  zal  de 
vertraging  in  de  verbinding  de  rechtstreekse  aansturing  van  de  UAV  niet  extreem 
bemoeilijken.  De  vliegeigenschappen  (‘handling  qualities’)  van  de  UAV  spelen  wel  een 
rol:  slechte  basiseigenschappen  kunnen  onder  invloed  van  vertraging  overgaan  in 
onacceptabele  vliegeigenschappen.  Het  heeft  dus  zin  om  op  zoek  te  gaan  naar  UAV’s 
met  zo  goed  mogelijke  vliegeigenschappen:  deze  UAV’s  verdragen  hogere 
vertragingswaarden.  Dit  uiteraard  vooral  als  de  betreffende  UAV  direct  door  een 
operator  wordt  aangestuurd. 

Bij  gebruik  van  geostationaire  satellieten  loopt  de  vertraging  doorgaans  zover  op  dat  de 
UAV-operator,  ondanks  alle  compensatietechnieken  die  beschikbaar  zijn,  vooral  in  een 
supervisierol  (programmeren  en  bewaken  van  de  route)  zal  opereren.  Alleen  de 


ONGERUBRICEERD 


ONGERUBRICEERD  |  TNO  report  |  TNO-DV3  2005  A054 


5/50 


sensoren  van  de  UAV,  die  een  iets  minder  kritische  rol  vervullen,  zullen  ook  in  dit 
geval  meestal  nog  rechtstreeks  door  een  operator  aangestuurd  kunnen  worden. 
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Summary 


Purpose: 

UAVs  (Unmanned  Aerial  Vehicles)  are  remotely  controlled  from  a  GCS  (Ground 
Control  Station).  The  nature  of  the  connection  between  the  two  is  such  that  some  delay 
of  the  control  signals  from  GCS  to  UAV  and  of  feedback  (telemetry  and  sensor 
information)  is  to  be  expected.  These  delays  may  have  an  impact  on  the  performance  of 
the  UAV  operator. 

The  Royal  Netherlands  Air  Force  (RNLAF)  is  considering  the  potential  acquisition  of 
UAVs.  In  this  process  it  is  of  importance  to  know  the  kind  of  constraints  that  arise  from 
delays  resulting  from  the  various  possible  ways  of  UAV  operations,  and  the  potential 
solutions  that  may  be  applied  in  a  candidate  for  acquisition.  TNO  performed  a  study 
specifically  aimed  at  signal  latency.  Other  aspects  of  the  data  link  (e.g.,  bandwidth, 
continuity  of  the  transmission,  noise  and  trustworthiness)  were  not  part  of  the 
commission. 

Method: 

We  examined  literature  on  sources  of  delay  (and  performed  some  calculations  in  this 
area).  Additionally,  we  studied  literature  concerning  the  effects  of  delays  on  human 
performance,  and  literature  on  potential  solutions  for  the  delay  problem. 

Results: 

Depending  on  the  situation,  delays  turn  out  in  the  range  of  100  to  1600  ms  (and  even 
more).  This  is  a  considerable  amount  given  that  100  ms  delay  usually  leads  to 
measurable  degradation  of  human  performance.  Delays  of  about  250-300  ms  quite  often 
lead  to  unacceptable  airplane  handling  qualities.  Techniques  such  as  filtering  and 
predictive  displays  exist  that  may  extend  the  range  of  acceptable  delays  up  to  about  400 
ms  or  more.  This  holds  for  the  UAV  control.  Control  of  UAV  sensors  is  considerably 
less  critical  and  the  use  of  the  delay  compensation  techniques  will  probably  lead  to 
acceptable  performance  with  delays  up  to  1000  ms  and  beyond. 

Conclusions: 

Signal  delays  between  GCS  and  UAV  will  not  significantly  impede  direct  control  of  the 
UAV  in  the  case  of  line-of-sight  connections  or  while  using  relay  stations  on  or  close  to 
the  earth.  Handling  qualities  of  the  UAV  play  a  role:  poor  handling  qualities  of  the 
UAV  may  transform  into  unacceptable  handling  qualities  under  the  influence  of  delays. 
Therefore,  it  is  useful  to  look  for  UAVs  with  good  handling  qualities:  They  can 
withstand  higher  delays.  This  is  of  interest  first  and  foremost  if  the  UAV  in  question  is 
directly  controlled  by  a  human  operator. 

When  geostationary  satellites  are  used,  delays  will  generally  be  so  high  that  a  UAV 
operator,  all  the  compensation  methods  notwithstanding,  will  just  operate  in  a 
supervisory  role  (programming  and  overseeing  the  route).  Only  the  control  of  the 
sensors  of  the  UAV,  less  critical  than  the  control  of  the  vehicle,  may  be  handled  directly 
by  an  operator. 
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1  Introduction 


UAV  ‘pilot’  receives  air  medal 

An  1 1th  Reconnaissance  Squadron  unmanned  aerial  vehicle  operator  was  recently 
awarded  the  Air  Force  Aerial  Achievement  Medal  for  safely  landing  a  UAV  after 
its  engine  seized  150  miles  from  the  ground  control  station  at  Mostar  Air  Base, 
Bosnia-Herzegovina. 

Capt.  Greg  Harbin  was  able  to  remotely  glide  the  unmanned  aircraft  for  about  30 
miles,  avoiding  populated  areas  and  manoeuvring  the  UAV  to  the  airfield  where  it 
could  be  safely  recovered.  The  landing  was  made  more  difficult  because  the  nose 
camera ,  used  as  the  primary  pilot  camera ,  iced  over  during  the  descent  and  the 
aircraft  was  being  controlled  by  its  satellite  link ,  which  causes  a  delay  in  aircraft 
control  response  time  (Airman,  1998). 

Officials  release  Global  Hawk  accident  report 

Air  Force  investigators  have  determined  mechanical  failure  caused  an  Air  Force 
RQ-4A  Global  Hawk  aircraft  to  crash  July  10  during  a  surveillance  mission 
supporting  Operation  Enduring  Freedom. 

The  Global  Hawk,  an  unmanned  reconnaissance  aircraft,  was  destroyed  upon 
impact  in  an  unpopulated  area  in  the  U.S.  Central  Command  area  of 
responsibility.  No  one  was  injured  in  the  accident.  The  aircraft  was  assigned  to 
the  12th  Expeditionary  Reconnaissance  Squadron,  but  its  parent  unit  is  the  9th 
Reconnaissance  Wing  at  Beale  Air  Force  Base,  California. 

According  to  an  Air  Combat  Command  Accident  Investigation  Report  released 
Dec.  6,  the  primary  cause  of  the  accident  was  the  failure  of  a  single  fuel  nozzle  in 
the  high-flow  position  that  eventually  caused  internal  failure  of  the  engine.  The 
aircraft  crashed  during  an  attempted  emergency  landing.  (Air  Force  Link,  2002) 


The  press  stories  above  tell  a  story  about  mishaps  with  two  fairly  distinct  Unmanned 
Aerial  Vehicles  (UAVs),  the  General  Atomics’  Predator  and  Northrop  Grumman’s 
Global  Hawk ,  respectively7.  The  former  is  a  so-called  MALE  (Medium  Altitude  Long 
Endurance)  UAV,  the  latter  is  a  HALE  (High  Altitude  Long  Endurance)  UAV.  Both  of 
these  UAVs  have  been  generally  very  successful  in  the  recent  US  military  campaigns  in 
Afghanistan  and  Iraq. 

Apart  from  their  dissimilar  capabilities,  the  Predator  and  the  Global  Hawk  are  operated 
in  completely  different  ways  as  well.  The  Predator  can  be  controlled  by  a  pilot  just  like 
a  manned  airplane  using  yoke  and  pedals,  whereas  the  Global  Hawk  is  almost 
completely  automated.  It  is  possible  to  override  the  Global  Hawk’s  airspeed,  altitude 
and  heading,  but  that’s  about  it.  There  is  no  low-level,  pilot-type  of  control  for  the 
Global  Hawk.  This  means  that  there  is  not  much  that  the  controller  can  do  in  case  of 
abnormal  flight  conditions.  The  control  algorithms  of  the  Global  Hawk  have  to  be 
sufficiently  robust  to  deal  with  unpredictable  emergencies. 


I  assume  the  reader  to  be  familiar  with  UAVs.  For  an  introduction  to  UAVs  see  De  Vries,  Van  der  Veen 
and  Krabbendam  (2000). 
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As  the  above  story  seems  to  show,  the  human  UAV  controller  may  succeed  where  the 
autonomous  UAV  fails.  The  story  also  hints  at  the  problem  of  control  in  situations  with 
delays.  It  has  been  known  for  long  that  even  small  delays  in  control-loops  may  lead  to 
large  control  difficulties.  This  problem  is  rather  native  to  the  UAV  field  because 
operator  and  UAV  may  be  separated  by  large  distances  connected  by  datalinks  with 
large  latencies. 

This  report  presents  the  results  of  a  literature  study  on  the  issue  of  UAVs  and  latency. 
Latency  is  not  only  a  problem  for  the  control  of  the  UAV  platform  itself.  Most  UAVs 
carry  sensors,  particularly  cameras,  which  have  to  be  controlled  as  well.  This  report  will 
also  deal  with  sensor  control.  Given  the  trend  towards  increasing  autonomy  in  UAVs, 
camera  control  with  delay  in  the  loop  may  even  be  a  more  important  issue  than  platform 
control. 

Other  aspects  of  UAV  datalinks  (e.g.,  bandwidth,  continuity  of  the  transmission,  noise 
and  trustworthiness)  were  not  part  of  the  commission  and  were  therefore  not  studied.  A 
study  on  these  individual  aspects  or  their  combination  may  be  warranted,  though. 
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2  Sources  of  delay 

In  this  section  I  will  discuss  the  various  sources  of  delays,  leading  to  an  assessment  of 
typical  delays  to  be  expected  in  a  few  example  systems.  Sources  of  delay  that  will  be 
discussed  are: 

•  Signal  transport; 

•  Datalink  electronics; 

•  Encryption; 

•  Compression; 

•  Error  correction; 

•  Synchronization; 

•  Computations. 

2.1  Signal  transport 

2.1.1  Electromagnetic  wave  propagation 

The  signals  involved  in  sending  information  between  the  UAV  and  the  GCS  are 
transmitted  using  electromagnetic  waves.  These  waves  have  a  finite  propagation  speed, 
which  in  vacuum  is  2.9979x1 08  m/s.  In  copper  and  glass-fiber  cables  the  speed  is 
typically  about  2/3  of  this  speed.  Due  to  the  finite  speed,  a  delay  occurs  in  any 
transmission,  depending  on  the  distance  between  the  transmitter  and  receiver  and  the 
route  followed  by  the  signals. 

If  it  were  possible  for  the  waves  to  crawl  along  the  entire  surface  of  the  earth,  a  round- 
trip  would  take  about  132  ms.  Although  this  may  seem  a  rather  small  number  we  will 
see  that  it  is  far  from  negligible. 

Propagation  of  waves  used  for  radio  and  TV  signals  (in  the  HF,  VHF  and  part  of  the 
UHF  band)  is  usually  divided  in  three  types:  ground-wave  propagation,  sky-wave 
propagation  and  space-wave  propagation. 

The  first  category  comes  closest  to  the  ‘crawling’  type  of  propagation.  Mainly  by  means 
of  diffraction,  the  waves  follow  the  contour  of  the  earth  and  manage  in  this  way  to 
travel  beyond  the  visual  horizon.  However,  due  to  the  interaction  with  surface  elements 
the  signal  strength  gets  attenuated  and  therefore  the  ‘radio  horizon’  is,  as  a  general  rule- 
of-thumb,  about  30  %  farther  than  the  visual  horizon.  An  exception  occurs  when  a 
temperature  inversion  is  present  (see  Figure  1).  In  that  case,  the  range  is  extended  by 
several  hundreds  of  kilometers,  sometimes  by  eight  times  the  visual  range.  The 
occurrence  of  this  phenomenon  is  rather  unpredictable  and  it  is  therefore  not  very  useful 
operationally. 
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Figure  1  The  effect  of  a  temperature  inversion  on  wave  propagation. 

In  sky-wave  propagation  the  signals  leave  the  surface  of  the  earth  and  are  refracted  by 
some  of  the  layers  of  the  ionosphere  (see  Figure  2).  The  ionosphere  has  at  least  four 
layers  at  day,  the  so-called  D,  E,  FI  and  F2  layers.  At  night,  the  D  and  E  layers 
disappear  and  the  FI  and  F2  layers  merge  into  one  F  layer.  For  refraction,  the  F  layer,  at 
150-400  km  above  the  earth  surface,  is  of  most  importance. 


100  MHz 


The  refraction  by  the  ionosphere  depends  on  the  frequency  of  the  waves  and  on  the 
angle  of  incident.  Beyond  a  critical  angle  and  frequency  the  waves  are  not  refracted 
back  to  the  earth  and  go  spaceward  (space-wave  propagation). 

Usually,  there  is  a  gap  between  the  areas  that  can  be  reached  by  the  ground  waves  and 
the  sky  waves  (see  Figure  3).  These  areas,  called  skip  zones,  can  be  hundreds  of 
kilometers  long. 
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Figure  3  The  origin  of  unreachable  areas,  the  ‘skip  zones’. 


Because  the  position  and  size  of  the  skip  zone  varies  with  time  of  day,  season  and  with 
the  number  of  sun  spots,  over-the-horizon  transmission  without  relays  is  not  very  useful 
for  a  reliable  GCS-to-UAV  connection.  Therefore,  three  types  of  links  for  control  of 
UAV  and  sensor  are  useful:  line-of-sight,  satellite  and  other  relay  communications. 
They  are  discussed  in  the  following  sections. 


2.7.2  Line-of-Sight  links 

As  discussed  in  the  previous  paragraph,  line-of-sight  links  are  limited  to  a  range  of 
about  the  visual  horizon.  The  HF  and  lower  VHF  frequencies  allow  for  a  sometimes 
considerable  extension  of  this  range,  but  for  the  UHF  and  higher  bands  (L,  S,  C,  X,  Ku, 
Ka),  wave  propagation  follows  more  or  less  the  rules  of  geometrical  optics,  i.e.  the 
visual  horizon  is  roughly  the  upper  limit. 
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Figure  4  Distance  along  the  surface  of  the  earth  ( S )  and  through  the  atmosphere  (D)  between  a 

transmitter  at  height  ht  and  a  receiver  at  height  hr.  R  denotes  the  earth  radius.  Note:  some 
authors  increase  the  radius  with  a  factor  of  4/3  to  roughly  incorporate  the  refraction  of  the 
electro-magnetic  waves  by  the  earth’s  atmosphere.  I  have  not  done  this  here. 

It  is  relatively  straightforward  to  make  line-of-sight  calculations.  Using  the  symbols 
defined  in  Figure  4  the  distance  along  the  earth  surface  between  transmitter  and  receiver 
is  given  by: 


S  =  R 


R  R 

arccos - +  arccos  - 


V 


R  +  hr 


R  +  ht 


(i) 


The  distance  the  transmitted  signals  travel  is  given  by: 


D  =  -,/(/? +  /1,.)2-/? 2  +>/(/?  +  /i,)2 -/?2  .  (2) 

Using  Equations  1  and  2,  we  can  calculate  some  typical  distance  and  delay  values  (see 
Table  1).  It  follows  that  line-of-sight  links  are  rather  limited  in  range  if  the  UAV  needs 
to  be  controlled  while  flying  close  to  ground  level.  If  the  UAV  operates  at  higher  flight 
levels,  the  line-of-sight  distance  may  increase  to  a  more  strategically  sized  value  (see 
Figure  1).  For  all  distances,  even  in  the  case  of  the  UAV-UAV  relay  (last  line  of  Table 
1),  the  delay  is  low,  less  than  3.5  ms. 

It  should  be  noted  here  that  the  calculations  assume  a  perfectly  spherical  and  smooth 
earth,  which  isn’t  the  case  in  real  life.  In  mountainous  areas  the  link  range  can  be 
drastically  less  than  shown  here,  due  to  ‘shadowing’  and  interference. 
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Table  1  Maximum  distances  between  various  transmitters  and  receivers  and  time  necessary  for  signal 
propagation  between  them. 


Description 

Height  GCS 
(m) 

Height  UAV 
(m) 

Earth 

distance 

(km) 

Signal 

distance  (km) 

Time 

(ms) 

Horizon 

average 

human 

1.8 

0 

4.8 

4.8 

0.0 

Horizon 

mobile  mast 

15 

0 

13.8 

13.8 

0.0 

Horizon 

321 

0 

64.0 

64.0 

0.2 

Vaalserberg 

idem  +  low 

MALE  UAV 

321 

900 

171.1 

171.1 

0.6 

idem  +  high 

MALE  UAV 

321 

7500 

373.2 

373.4 

1.2 

idem  +  low 

HALE  UAV 

321 

13500 

478.6 

479.2 

1.6 

idem  +  high 

HALE  UAV 

321 

19500 

562.1 

563.1 

1.9 

hi  HALE  to  hi 

HALE  UAV 

19500 

19500 

996.3 

998.3 
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Figure  5  Line-of-sight  ranges  from  the  Vaalserberg,  the  highest  hill  in  the  Netherlands.  Black  circle:  the 
horizon.  Blue  circles:  mark  the  area  in  which  MALE  UAVs  (operating  height  ca.  900-7500  m) 
are  visible.  Red  circles:  mark  the  area  in  which  HALE  UAVs  (ca.  13500-19500  m)  are  visible. 
All  ranges  are  based  on  a  round-earth  assumption  without  obstructions  (buildings,  mountains). 


ONGERUBRICEERD 


ONGERUBRICEERD  |  TNO  report  |  TNO-DV3  2005  A054 


15/50 


2. 1.3  Satellite  communication 

Due  to  the  limited  range  of  line-of-sight  datalinks,  relay  by  communication  satellite  is 
the  default  alternative  for  over-the-horizon  communication. 

Using  a  slight  modification  of  Equations  1  and  2  and  symbols  defined  in  Figure  6  we 
find: 


and 


/ 


S  =  R 


arccos 


v 


R 

R  +  hG 


+  2 •  arccos 


R 

R  +  hs 


+  arccos 


R  ' 
R  +  hu  > 


(3) 


D  =  \/(«  +  Ag)2-«2  +  2-J{R  +  hs)2-R2  +y[{R  +  h^)2 -fl2  .  (4) 


Figure  6  Distance  along  the  surface  of  the  earth  (S  =  SGS  +  Ssu)  and  through  the  atmosphere  ( D  =  Dc,s  + 
Gsu)  between  a  GCS  at  height  hG ,  a  Satellite  at  hs  ,and  a  UAV  at  height  hu.  R  denotes  the  earth 
radius. 


2. 1.3. 1  Low-Earth  Orbit  satellites 

Low-earth  orbit  (LEO)  satellites  generally  orbit  at  a  height  between  200  and  800  km, 
usually  at  large  angles  with  the  equator.  At  these  heights,  a  full  orbit  takes  about  1.5 
hour.  As  Figure  7  shows,  LEOs  can  ‘see'  entire  continents,  but  they  lose  sight  of  a 
given  ground  station  in  7-13  minutes  and  they  don’t  return  to  a  given  point  within 
several  hours  or  days  (see  Figure  8).  LEOs  can  therefore  not  be  used  for  sustained 
communication.  To  that  end,  an  extensive  and  hence  very  expensive  network  of  LEO 
satellites  is  necessary.  The  GPS  navigation  system,  for  instance,  uses  27  satellites.  The 
Irridium  satellite  telephony  system  was  supposed  to  operate  with  72  satellites. 
Calculations  showed  that  if  only  45  of  those  are  in  operation,  a  multiple-hop  transport 
of  a  signal  should  have  a  latency  of  maximal  178  ms  (Nichols,  1998).  Simple  relays 
using  LEO  satellites  have  a  moderate  signal  delay  between  1  and  24  ms  (see  Table  2). 


ONGERUBRICEERD 


ONGERUBRICEERD  |  TNO  report  |  TNO-DV3  2005  A054 


16/50 


Figure  7  Visual  horizon  (in  Mollweide  projection)  of  a  few  low  earth  orbit  satellites.  Red:  the  KVR- 
1000  satellite  at  200  km.  Green:  the  SIS  satellite  at  470  km.  Blue:  the  EarlyBird  satellite  at 
680  km. 


Figure  8  A  low-earth  orbit  satellite  in  a  polar  orbit  at  a  height  of  200  km  (orthographic  projection).  The 
green  curve  marks  the  positions  above  which  the  satellite  flew  in  a  time  span  of  9.3  hours.  The 
area  hatched  in  red  gives  an  indication  of  the  area  covered  during  the  trip  and  the  amount  of 
overlap  between  successive  laps.  Each  of  the  horizontal  lines  indicates  the  longitudinal  view. 
Horizontal  lines  are  separated  in  time  by  one  minute. 
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Table  2  Distances  and  delay  times  involved  in  low-earth  orbit  satellite  relay  calculated  using  Equations 
3  and  4.  All  rows  except  the  last  contain  various  GCS-UAV  configurations.  The  last  row 
describes  a  UAV-satellite-UAV  link. 


Satellite  height 
(km) 

Height 

GCS 

m 

Height 

UAV 

(m) 

Min.  dist. 

(km) 

Max.  dist. 

(km) 

Min.  time 

(ms) 

Max.  time 

(ms) 

200 

15 

100 

399.9 

3269.1 

1.3 

10.9 

470 

15 

100 

939.9 

5036.2 

3.1 

16.8 

680 

15 

100 

1359.9 

6095.1 

4.5 

20.3 

680  +  low  MALE  UAV 

15 

900 

1359.1 

6166.5 

4.5 

20.6 

680  +  high  MALE  UAV 

15 

7500 

1352.5 

6368.8 

4.5 

21.2 

680  +  low  HALE  UAV 

15 

13500 

1346.5 

6474.6 

4.5 

21.6 

680  +  high  HALE  UAV 

15 

19500 

1340.5 

6558.5 

4.5 

21.9 

680  +  hi  HALE  to  hi  HALE 

19500 

19500 

1321.0 

7043.8 

4.4 

23.5 

2. 1.3. 2  Geostationary  satellites 

Most  of  the  current  over-the-horizon  UAV  operations  are  handled  with  geostationary 
satellites.  Geostationary  satellites  are,  as  their  name  suggests,  stationary  with  respect  to 
the  earth.  Because  the  earth  is  rotating,  geostationary  satellites  must  have  a  circular 
orbit  lying  in  a  plane  through  the  equator.  The  radius  of  the  orbit  is  about  42,147  km, 
which  is  35,768  km  above  mean  sea  level. 

Due  to  their  altitude,  geostationary  satellites  can  cover  large  parts  of  the  earth  (see 
Figure  9).  Only  the  polar  areas  escape  attention.  A  GCS  and  a  UAV  that  are  within  the 
same  service  area  can,  at  least  in  principle,  receive  each  other  using  the  satellite  as 
relay.  Of  course,  at  the  boundaries  of  their  ranges  signal  reception  gets  considerable 
worse  due  to  the  oblique  incident  angle  and  the  longer  pathway  through  the  atmosphere. 
Also  note  that  wide  area  beams  used  by  the  satellites  often  cover  only  about  20%  of  the 
total  possible  area.  If  a  satellite  uses  spot  beams,  the  area  covered  will  be  considerably 
smaller.  Generally,  more  than  three  geostationary  satellites  will  be  necessary  to  cover 
the  whole  earth. 


Figure  9  The  surfaces  ‘seen’  by  three  geostationary  satellites  positioned  at  25°  W  (red),  145°  W  (blue, 
split  in  two  parts)  and  85°  E  (green).  The  earth  and  the  marked  areas  are  in  Mollweide 
projection. 
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Table  3  Signal  travel  distances  and  times  calculated  using  Equations  3  and  4.  Minimum  distances  occur 
when  GCS  and  UAV  are  both  straight  below  the  satellite.  Maximum  distances  occur  as 
sketched  in  Figure  6.  Round-trip  delays  are  obtained  by  doubling  the  indicated  times.  The 
values  here  and  in  some  other  tables  are  expressly  shown  with  an  absurd  number  of  digits  to 
highlight  the  minimal  differences  between  them. 


Description 

Height 

GCS 

(m) 

Height 

UAV 

(m) 

Min.  dist. 

(km) 

Max.  dist 

(km) 

Min.  time 

(ms) 

Max.  time 

(ms) 

Mobile  mast 

15 

0 

71536.0 

83335.9 

238.6 

278.0 

Hill 

321 

0 

71535.7 

83386.1 

238.6 

278.1 

idem  +  low  MALE  UAV 

321 

900 

71534.8 

83493.2 

238.6 

278.5 

idem  +  high  MALE  UAV 

321 

7500 

71528.2 

83695.5 

238.6 

279.2 

idem  +  low  HALE  UAV 

321 

13500 

71522.2 

83801.3 

238.6 

279.5 

idem  +  high  HALE  UAV 

321 

19500 

71516.2 

83885.2 

238.6 

279.8 

2.1.4  Relay 

2. 1.4.1  Fixed  repealers 

For  civilian  over-the-horizon  communication  the  signals  are  often  relayed  by  means  of 
one  or  more  fixed  relay  stations.  The  transport  delays  caused  by  this  method  are  very 
low  (about  1  ms  for  every  300  km).  However,  due  to  their  stationary  nature,  these  relay 
stations  are  not  of  much  practical  use  for  military  operations,  since  the  opponents  are 
not  likely  to  provide  the  necessary  facilities.  It  may  be  used  to  connect  MOB,  AOB, 
FOB  and  HQ  in  more  friendly  areas,  though. 

2. 1.4. 2  Pseudolytes 

Pseudolytes  are  self-propelled  flying  platforms  that  serve  in  the  same  role  as  satellites, 
albeit  at  a  considerable  lower  altitude  (typically  60.000  feet)  and  with  less  endurance 
than  either  low-earth  orbit  or  geostationary  satellites. 


Figure  10  The  AeroVironment  Helios  taking  off  from  the  Pacific  Missile  Range  Facility,  Kauai,  Hawaii. 
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The  role  of  UAVs  as  pseudolytes  is  still  in  a  concept  phase.  As  a  follow-up  to  the 
NASA  Pathfinder  prototypes,  the  solar-powered  Helios  (see  Figure  10)  built  by 
AeroVironment  is  planned  to  have  an  endurance  of  six  months  while  flying  at  an 
altitude  of  60,000-100,000  feet.  Among  other  uses,  it  is  advertised  as  a  communications 
platform. 

Other  R&D  groups  are  adapting  existing  UAVs  to  their  needs.  In  their  Extendor 
program,  QinetiQ  (the  commercial  branch  of  the  former  UK  defense  research  laboratory 
DER A)  is  exploring  the  possibilities  of  UAV  relays  using  a  Predator  UAV.  In  an 
experiment,  the  modified  Predator  successfully  relayed  digital  messages  from  a 
Forward  Air  Controller  to  a  Jaguar  HUD.  The  UAV  also  relayed  video  to  a  F- 16  and 
F-14  (Hardy,  2002). 

Given  its  endurance  and  operating  height,  the  Predator  (24-30  hours,  45,000  ft)  seems  a 
less  likely  candidate  for  communication  relay  than  the  Global  Hawk  (40  hours,  65,000 
ft).  The  Helios  has  a  larger  endurance  than  the  other  systems,  but  doesn’t  have,  at  this 
time,  sufficient  payload  capacity  and  electrical  power  to  carry  a  large  communication 
relay  platform. 

At  the  planned  heights  (up  to  100,000  ft),  UAV  relays  yield  a  maximum  range  of 
approximately  1300  km.  The  one-way  signal  delay  caused  by  this  distance  is  4.3  ms.  If 
the  platform  lives  up  to  its  promise  this  low  value  would  make  it  an  ideal  relay  station 
for  commanding  UAVs.  It  can  loiter  above  a  fixed  position  on  earth  like  the 
geostationary  satellites,  but  it  can  be  easily  maneuvered  to  other  areas  when  the  need 
arises. 


2.2  Datalink  electronics 

The  analog  electronics  that  build  up  the  datalink  introduce  a  certain  amount  of  latency. 
One  source  (Hall,  Hart,  &  Wasel,  2001)  claims  that  most  current  UAV  systems  utilize 
an  analog  datalink  that  causes  a  minimal  latency  of  about  100  ms,  whereas  legacy  RF 
links  used  to  have  a  latency  of  40  ms.  They  warn  that  modern  digital  links  will  have 
higher  latencies  than  their  analog  counterparts. 

Quite  often,  there  are  various  steps  between  the  UAV  and  its  GCS.  For  instance,  with 
the  French  Sperwer  UAV-system,  in  use  with  the  Dutch  army’s  101st  Remotely  Piloted 
Vehicles  Battery,  the  signals  from  the  GCS  are  sent  by  glass-fiber  cable  to  a  Ground 
Data  Terminal  (GDT),  which  transmits  them  to  the  UAV  (see  Figure  1 1).  The  rational 
is  that  the  GDT  works  like  an  electromagnetic  beacon,  making  it  vulnerable  to  attack. 
Therefore,  it  is  better  to  separate  the  two  functions  of  control  and  communication  by  a 
considerable  distance. 

It  is  safe  to  assume  that  every  extra  datalink  that  gets  in  between  the  GCS  and  the  UAV 
adds  its  own  latency.  Hence,  communications  systems  built  like  the  Sperwer  will  have 
electronics  related  latencies  of  at  least  200  ms. 
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Figure  1 1  Top:  an  overview  of  a  typical  Sperwer  battery  configuration.  Bottom  left:  the  interior  of  the 
Sperwer  GCS.  Bottom  right:  The  Sperwer  GDT. 


2.3  Encryption/Decryption 

Signals  from  and  to  the  UAV  should  be  encrypted  if  a  datalink  is  used  that  is  easily 
detected  and  compromised  such  as  the  UHF-band,  which  generally  doesn’t  require 
directional  antennas.  The  usage  of  narrow  beams  and  frequency  hopping  schemes 
increases  security.  Encryption  will  nevertheless  be  necessary  to  increase  security  and  to 
provide  authorization  of  control  (by  means  of  authentication). 


Luckily,  encryption  does  not  add  much  to  the  datalink’ s  latency.  Blaze  (1996) 
encrypted  high  bandwidth  links  using  a  Pentium  90  MHz  computer  with  a  latency  of 
about  20  ms.  Nowadays  computers  or  dedicated  hardware  should  be  able  to  improve  on 
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this  by  an  order  of  magnitude.  One  can  expect  a  1-2  ms  delay.  However,  each  link  (up 
and  down)  needs  both  encryption  and  decryption.  Therefore,  roundtrip  latency  due  to 
encryption  will  probably  be  in  the  order  of  4-8  ms. 

2.4  Compression/decompression 

Conservation  of  bandwidth  is  of  great  importance.  The  radio  spectrum  is  a  scarce 
resource  and  most  of  the  available  frequency  bands  are  already  in  use,  regulated  by 
multi-national  treaties  and  organizations  (see,  e.g.,  FCC,  2002).  Given  the  successful 
use  of  UAVs  in  the  Balkan,  Afghan,  and  Iraqi  theatres  and  the  plans  to  convert  1/3  of 
the  US  deep-strike  force  into  unmanned  platforms  (US  Senate  budget  proposal  2000),  it 
is  clear  that  the  number  of  datalinks  will  increase.  The  military  has  to  conserve  its 
allotted  space  and  hence  must  turn  to  data  compression  and  other  bandwidth-saving 
techniques. 

In  the  UAV  case,  probably  only  the  high  bandwidth  downlink  will  need  to  be 
compressed  (video  and  radar  data).  The  uplink,  consisting  mostly  of  low  data-rate 
control  commands  does  not  need  this.  In  this  section  we  will  concentrate  on  the 
compression  of  video-material. 


There  are  many  different  video  compression  methods.  The  compression  of  choice  at  the 
moment  is  MPEG-2  which  will  be  succeeded  by  MPEG-4  shortly. 


MPEG  is  a  ‘lossy’  compression  method,  which  means  that  the  original  and  the 
compressed  and  decompressed  version  of  the  original  are  not  the  same.  The  difference 
may  or  may  not  be  noticeable,  depending  on  the  quality  setting.  MPEG  is  akin  to  the 
JPEG  standard  for  still  imagery,  but  it  not  only  exploits  redundancy  within  a  single 
movie  frame  but  it  also  uses  redundancy  between  several  successive  frames.  In  order  to 
do  this.  MPEG  has  to  buffer  several  video  frames,  which  means  that  without  performing 
any  calculations  at  least  100  ms  or  so  are  already  lost. 


MPEG-2  uses  JPEG-like  coding  for  so-called  I-frames,  and  can  additionally  calculate 
P-frames  (forward  Predicted  frames,  based  on  previous  I  and  or  P  frames)  and  B-frames 
(Bi-directional  interpolated  frames,  based  on  previous  and  future  I  and  P  frames)  to 
decrease  data  rate  and/or  to  improve  quality.  Chow  (2005)  claims  compression  latencies 
to  be  200  ms  to  400  ms,  200  ms  to  500  ms,  and  400  ms  to  850  ms  for  movies  with  I- 
frames,  IP-frames,  and  IBP-frames,  respectively. 


Minerva  Networks  (2001)  produces  a  hardware  MPEG-compressor  for  video- 
conferencing  applications.  They  claim  a  ‘record  breaking  end-to-end  latency’  of  150 
ms.  Given  that  they  don’t  use  full  resolution  video  one  should  scale-up  their  value  by  a 
factor  of  about  2.5  to  obtain  a  latency  value  for  full-screen  video.  This  amounts  to  about 
375  ms.  MPEG  compression  is  generally  much  more  time-consuming  than  MPEG 
^compression,  hence  this  end-to-end  value  (compression  plus  decompression)  is  close 
to  Chow’s  range  of  values.  We  will  use  this  value  for  our  estimates. 
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2.5  Error  correction 

Error  correction  is  necessary  to  prevent  the  potentially  disastrous  situation  of  distorted 
control  commands  being  executed  by  the  UAV. 

In  the  TCP/IP  protocol  used  for  Internet  communications,  checksums  are  applied  to 
detect  data  corruption.  When  a  data  packet  is  received  corrupt,  a  retransmission  is 
requested.  Of  course,  this  implies  a  relatively  enormous  amount  of  latency. 

Other  protocols,  like  UDP/IP,  simply  drop  the  corrupted  packet.  This  is  generally  no 
problem  for  data  that  is  not  essential,  like  a  couple  of  frames  from  a  video  or  sound 
transmission,  but  for  control  signals  this  is  probably  not  a  good  idea.  In  this  case  error 
correction  is  necessary. 

By  means  of  clever  encoding  it  is  often  possible  to  detect  data  corruption  and  correct  for 
it.  A  certain  amount  of  extra  data  (depending  on  the  amount  of  errors  one  would  like  to 
be  able  to  correct)  needs  to  be  send  along  with  the  data.  Hardware  solutions  exist  that 
have  a  latency  of  less  than  1.5  ms  (see,  e.g.,  4i2i  communications  Ltd,  2001). 


2.6  Synchronization 

In  a  process  consisting  of  several  sequential  free-running  sub-processes  the  lack  of 
synchronization  is  a  potential  source  of  delays.  When  two  processes  are  not 
synchronized  but  take  about  the  same  time,  the  second  process  has  to  wait  for  the  first 
process  to  finish.  In  a  worst  case  scenario  this  idle  waiting  can  take  as  much  as  the  total 
process  cycle  time  of  the  first  process.  On  average,  half  the  processing  time  will  be 
spent  waiting. 

It  is  hard  to  tell  in  which  of  the  stages  of  downlink  and  uplink  unsynchronized  processes 
appear.  It  is  at  least  very  likely  that  the  image  sensor  and  the  monitor  its  image  is 
displayed  on  are  unsynchronized.  Given  a  refresh  rate  of  60  Hz  this  implies  an  average 
delay  of  8  ms.  It  is  very  likely  that  more  synchronization  delays  will  be  present. 

2.7  Computation  delays 

The  presentation  of  the  Human-Computer  Interface  (HCI)  of  the  UAV  operator  takes 
some  time.  Symbology  has  to  be  updated,  sensor  footprints  need  to  be  calculated,  maps 
have  to  be  moved  and/or  rotated,  and  the  operator’s  input  has  to  be  processed. 

In  a  UAV  control  concept  like  that  of  the  Tactical  Control  Station  (TCS,  see  NSWC, 
1999)  a  hardware  abstraction  layer  shields  the  operator  from  the  specifics  of  the 
hardware  he  is  controlling.  The  purpose  of  this  is  to  increase  interoperability.  In  the 
future,  TCS  operators  should  be  able  to  control  various  kinds  of  UAVs  from  the  same 
console.  The  abstraction  layer  converts  outgoing  data  from  the  generic  console  to 
commands  specific  to  the  UAV,  and  does  the  same  for  the  incoming  telemetry  and 
sensor  data.  This  layering  will  add  to  the  delay. 

It  is  hard  to  give  any  hard  figures  for  the  delays  caused  by  all  this.  The  latency  will 
decrease  with  increasing  computing  power  and  is  probably  in  the  order  of  10-30  ms  at 
this  time. 
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2.8  Total  signal  delay 

We  are  now  ready  to  calculate  an  overall  latency.  In  Table  4,  latencies  are  summarized 
for  two  configurations,  a  short-range  line-of-sight  connection  and  an  over-the-horizon 
connection  using  geostationary  satellites  as  a  communications  relay.  It  is  a  ‘minimax’ 
approach,  which  means  that  we  estimate  the  worst  case  using  the  lowest  possible 
delays.  It  is  quite  possible  that  sub-optimal  hardware  and  software  results  in  higher 
latencies  than  presented  here. 

The  largest  estimate  is  1672  ms.  This  may  sound  rather  high,  but  it  may  actually  be  an 
underestimation.  For  instance,  there  are  claims  that  England-based  Dutch  UAV 
operators  who  were  controlling  a  camera  of  a  Predator  UAV  flying  above  Bosnia 
experienced  delays  of  up  to  6  s  (Muller,  2001).  It  is  hard  to  determine  how  precise  this 
estimate  is,  but  it  is  clear  that  our  estimate  of  about  2  s  is  in  the  right  ballpark  and 
certainly  not  too  high. 


Table  4  Estimates  for  minimum  and  maximum  latencies  (in  ms)  for  two  types  of  datalinks:  line-of-sight 
(LOS)  and  geostationary  satellite  relay  (GEO).  In  the  minimum  configuration  we  assume  no 
encryption,  compression  and  error  correction  and  use  the  lowest  estimation  of  electronics 
latency.  In  the  maximum  configuration  we  take  all  factors  into  account  and  assume  the 
presence  of  a  GDT  and  use  a  higher  estimation  of  electronics  latency.  We  also  assume  that  only 
the  downlink  is  compressed.  Although  these  numbers  appear  to  be  rather  precise  it  is  better  to 
regard  them  as  rough  estimates.  The  number  of  digits  has  been  increased  to  make  the  small 
contributions  to  the  total  delay  visible. 


GCS-UAV  configuration 

LOS 

GEO 

min 

max 

min 

max 

Tranceive 

40.0 

300.0 

80.0 

300.0 

Transport 

0.2 

3.3 

239.0 

281.0 

Encryption 

0.0 

4.0 

0.0 

4.0 

Compression 

0.0 

375.0 

0.0 

375.0 

Error  correction 

0.0 

1.5 

0.0 

1.5 

Synchronization 

8.0 

32.0 

8.0 

32.0 

Computations 

10.0 

30.0 

10.0 

30.0 

Uplink 

58.2 

370.8 

337.0 

648.5 

Downlink 

58.2 

745.8 

337.0 

1023.5 

Round  trip  total 

116.4 

1116.6 

674.0 

1672.0 
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3  Control  of  time-delay  systems 


Anyone  who  has  ever  tried  to  take  a  shower  in  a  house  with  a  long  distance  between 
shower  and  heater  knows  that  it  is  very  hard  to  get  the  temperature  of  the  shower  just 
right.  Due  to  the  large  delay  between  adjustment  of  the  shower  faucets  and  the 
corresponding  change  in  water  temperature  it  takes  a  lot  of  experimenting  and 
overshoots  (too  hot  or  too  cold)  before  a  comfortable  temperature  has  been  finally  set. 

Humans  are  notoriously  bad  in  dealing  with  systems  containing  delays.  Therefore, 
delays  have  been  a  popular  topic  for  research.  Ricard  (1994)  writes  in  his  extensive 
bibliography  on  manual  controls  with  delays:  “For  experimentalists,  delay  of  visual 
feedback  has  served  as  a  reliable  cue  over  the  past  four  decades  for,  in  the  limit,  lagging 
a  system’s  output  can  always  be  expected  to  effect  human  control  performance.” 

In  this  section,  a  short  introduction  to  the  theory  of  control  of  delayed  systems  will  be 
presented.  This  will  be  followed  by  some  general  psychophysical  data  on  delayed 
control  and  data  from  the  more  relevant  world  of  airplanes  and  simulators. 

3.1  Theory 

A  human  UAV  controller  can  be  seen  as  a  comparator  who  compares  desired  system 
output  with  the  actual  system  output  and  reacts  on  the  difference.  Of  course,  this  is  an 
absurd  oversimplification  ,  but  it  is  useful  to  show  the  source  of  the  control  problem. 

In  Figure  12  such  a  simplified  UAV  operator  can  be  seen.  At  t  =  0,  a  controlled 
parameter,  e.g.  speed  or  altitude  of  the  UAV,  needs  to  change  from  one  level  to  another 
level.  This  is  symbolized  by  the  ‘step’  box  in  the  diagram.  The  UAV  operator  compares 
this  desired  level  with  the  actual,  delayed  system  state  and  uses  the  difference  of  these 
levels  to  control  the  system  state.  The  difference  signal  multiplied  by  a  gain  factor  feeds 
directly  into  the  controlled  system,  which,  in  this  case,  is  modeled  by  a  simple 
integrator.  The  lower  part  of  the  figure  shows  the  responses  for  various  values  of  the 
operator  gain.  A  low  gain  leads  to  a  slow  response;  however,  it  reaches  the  desired  level 
in  a  very  stable  manner.  Higher  gains  will  bring  the  system  quicker  in  the  desired  range, 
but  they  tend  to  overshoot  the  goal  level  followed  by  (dampened)  oscillations.  Beyond  a 
certain  critical  gain  the  system  becomes  unstable.  Corrections  and  errors  get  into 
counter  phase  and  amplify  each  other. 

The  type  of  oscillations  and  the  critical  value  of  the  gain  are  of  course  totally  dependent 
on  the  specifics  of  the  system  (vehicle  transfer  function  and  delay).  Many  textbooks  on 
control  theory  deal  with  this  and  we  won’t  go  into  details  here.  The  example  was  used 
here  to  make  it  plausible  that  any  controller,  human  or  machine,  will  have  problems 
handling  delays  and  that  the  handling  problems  are  not  the  result  of  a  specific  human 
shortcoming. 


2  For  a  more  in-depth  discussion  of  human  controller  models,  see  (Hess,  1997). 
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Figure  12  Above:  a  Matlab  SimuLink  model  of  a  simple  controller-vehicle-delay  loop.  Below:  Output  of 
the  model  for  various  values  of  operator  gain.  The  higher  the  gain  gets,  the  less  stable  the 
system.  In  the  last  panel,  a  value  of  gain  is  reached  where  the  system  becomes  unstable. 


3.2  Equivalent  or  effective  platform  delay 

Apart  from  the  straightforward  time  delay  caused  by  finite  signal  transport  speed,  other 
types  of  delays  are  also  distinguished  (Smith  &  Sarrafian,  1986): 

To  a  pilot,  time  delay  is  the  dead  time  between  his  force  input  to  the  stick  and 
the  beginning  of  any  aircraft  response  or  output.  This  delay  can  come  from  a 
variety  of  sources  within  the  flight  control  system. 

A  system  that  reproduces  the  exact  shape  of  an  input  after  an  interval  of  dead 
time  is  defined  as  exhibiting  transport,  or  pure,  time  delay. 

. .  .the  majority  of  time  delay  in  modern  electronic  flight  control  applications  is 
not  caused  by  pure  time  delays.  Typically  the  complexity  of  modem  control 
system  design  strategies  results  in  cascading  numerous  dynamic  elements 
which  can  introduce  a  perceived  delay  in  the  initial  response  of  the  aircraft  to  a 
pilot  input.  This  form  of  time  delay  is  often  referred  to  as  “equivalent"  or 
“effective”  time  delay,  depending  on  the  measurement  method.  Each  method 
represents  an  approximation  of  the  dead  time  sensed  by  the  pilot. 


3.3  Psychophysical  data  not  directly  related  to  airplane  control 

From  the  huge  body  on  general  effects  of  time  delay  on  human  performance  only  a  few 
examples  will  be  presented  in  this  section.  For  an  extensive  bibliography  list,  see  Ricard 
(1994). 

As  Ricard  said,  you  are  sure  to  find  effects  from  delays.  However,  the  size  of  the  effects 
varies  wildly  in  the  literature.  Wargo  (1967)  studied  controller  performance  using 
pursuit  tracking  displays  (displays  that  both  show  the  value  to  be  achieved  by  the 
controller  and  the  actual  value)  versus  compensatory  displays  (displays  that  only  show 
the  error,  i.e.  the  difference  between  the  required  and  the  actual  value).  Pursuit  tracking 
displays  have  a  better  performance  than  compensatory  displays.  However,  pursuit 
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tracking  displays  are  more  influenced  by  delays  than  compensatory  displays,  a  result 
also  found  by  Conklin  (Conklin,  1957;  Conklin,  1960).  In  a  simple  tracking  task,  a 
delay  of  200  ms  lead  to  a  degradation  of  5%  of  the  performance  for  pursuit  displays 
whereas  compensatory  displays  were  only  degraded  by  2  %.  At  840  ms  these  numbers 
were  30  and  20%,  respectively. 

The  size  of  this  effect  does  not  differ  much  from  the  one  reported  by  Hill  (1976)  in  an 
entirely  different  experiment.  He  examined  the  control  of  a  manipulator  arm,  a  slave,  by 
a  master  arm  operated  by  a  human  controller.  The  results  show  that  the  time  needed  for 
execution  of  several  manipulation  tasks  grows  linearly  with  transmission  delay.  Hill 
also  noted  the  occurrence  of  a  ‘move-and-waif  strategy.  With  longer  delays  controllers 
often  wait  for  the  slave  arm  to  reach  a  certain  position  before  they  go  on  to  give  the  next 
motion  command.  After  about  300  ms  the  percentage  of  the  time  the  arm  is  busy 
moving  starts  to  decay  considerably,  from  about  50%  for  no  delay  and  45%  at  300  ms 
delay  up  to  10%  for  10  s  delay. 

The  above  mentioned  authors  reported  relatively  small  effects  for  delays  in  the 
hundreds  of  milliseconds.  On  the  other  hand,  Smith  and  Bowen  (1980)  found  that  66  ms 
delay  already  leads  to  measurable  eye-hand  coordination  deterioration.  Subjects 
typically  can  adapt  to  delays  lower  than  100  ms.  Other  researchers  report  stronger 
effects  of  delays  as  well.  For  instance,  Foulkes  and  Miall  (2000)  data  on  a  simple  2D 
tracking  task  show  subjects  to  have  errors  two  and  three  times  as  large  for  delays  of  200 
and  300  ms,  respectively,  compared  to  the  errors  in  the  non-delayed  condition. 

3.4  Airplane-related  data 

The  military  standard  MIL-F  8785  (US  Department  of  Defence,  1980)  sets  some  rules 
on  acceptable  delays.  It  defines  three  levels  of  flying  qualities  (p.  4): 

Level  1 :  Flying  qualities  clearly  adequate  for  the  mission  Flight  Phase 

Level  2:  Flying  qualities  adequate  to  accomplish  the  mission  Flight  Phase, 

but  some  increase  in  pilot  workload  or  degradation  in  mission 
effectiveness,  or  both,  exists 

Level  3:  Flying  qualities  such  that  the  airplane  can  be  controlled  safely,  but 
pilot  workload  is  excessive  or  mission  effectiveness  is  inadequate, 
or  both.  Category  A  Flight  Phases  can  be  terminated  safely,  and 
Category  B  and  C  Flight  Phases  can  be  completed. 

The  flight  phase  categories  A,  B,  and  C  mentioned  in  Level  3  are  defined  as  (p.  2-3): 

Category  A:  Those  non-terminal  Flight  Phases  that  require  rapid  maneuvering, 
precision  tracking,  or  precise  flight-path  control.  Included  in  this  category  are 

a.  Air-to-air  combat; 

b.  Ground  attack; 

c.  Aerial  recovery; 

d.  Reconnaissance; 

e.  In-flight  refueling  (receiver); 

f.  Terrain  following; 

g.  Antisubmarine  search; 

h.  Close  formation  flying. 
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Category  B:  Those  non-terminal  Flight  Phases  that  are  normally  accomplished  using 
gradual  maneuvers  and  without  precision  tracking,  although  accurate  flight-path  control 
may  be  required.  Included  in  this  category  are: 

a.  Climb; 

b.  Cruise; 

c.  Loiter; 

d.  In-flight  refueling  (tanker); 

e.  Descent; 

f.  Emergency  descent; 

g.  Emergency  deceleration; 

h.  Aerial  delivery. 


Category  C:  Terminal  Flight  Phases  are  normally  accomplished  using  gradual 
maneuvers  and  usually  require  accurate  flight-path  control.  Included  in  this  category 


are: 

a. 

Takeoff; 

b. 

Catapult  takeoff; 

c. 

Approach; 

d. 

Wave-off/go-around; 

e. 

Landing. 

For  an  airplane  to  be  classified  as  having  a  certain  level  of  handling  qualities,  a 
maximum  allowable  delay  is  specified  for  each  level  (p.42): 

Level  Allowable  delay  (ms) 

1  100 

2  200 

3  250 

Clearly,  given  the  description  of  level  3  above,  250  ms  is  the  absolute  maximum 
allowable  delay,  typical  for  very  poor  handling  qualities. 

Berry  (1985)  examined  the  effects  of  delays  added  to  the  lateral  (roll)  and  longitudinal 
(pitch)  stability  augmentation  system  (SAS)  of  the  NASA-modified  F-8  jet  for  landing 
and  formation  flying  tasks.  They  examined  effects  of  adding  between  20  and  200  ms  of 
delay  to  a  system  with  an  inherent  delay  of  130  ms  using  a  Cooper-Harper  flying 
qualities  rating  scale  (see  Figure  13).  In  the  pitch  axis,  in  calm  air,  spot  landings  were 
most  strongly  influenced  by  time  delay.  In  the  roll  axis,  in  calm  air,  formation  flying 
was  most  strongly  influenced  by  time  delay  (Cooper- Harper  ratings  increased  from  an 
average  of  about  3  to  6  if  the  incremental  time  delay  increased  to  140  ms).  However, 
when  landings  were  made  in  turbulence,  flying  qualities  in  pitch  were  only  slightly 
degraded,  whereas  in  roll  they  were  severely  degraded  (ratings  increase  from  3  to  8 
instead  of  from  3  to  about  4  when  landing  in  turbulence  instead  of  in  calm  air).  Berry 
found  the  MIL-F-8785  specification  to  be  reasonable  for  lateral  time  delays,  but  a  bit 
too  stringent  for  lateral  time  delays. 

Smith  and  Sarrafian  (1986)  counter  the  rigid  rules  of  the  MIL-F-8785  specification.  In 
their  research  they  found  acceptable  flying  qualities  to  be  strongly  dependent  on  the 
specific  task  (approach,  landing).  In  the  fly-by-wire  NASA  F-8  project,  an  equivalent 
latency  of  220  ms  yielded  a  pilot  rating  of  4  for  low  stress  tasks  and  8  for  high  stress 
tasks  on  a  Cooper-Harper  flying  qualities  scale.  They  found  that  for  planes  with  an 
equal  amount  of  total  equivalent  delay  (see  Section  3.2),  where  one  has  a  slow  feel 
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system  (force  feedback  on  the  control  stick)  and  the  other  a  fast  feel  system  each  with 
its  own  characteristic  equivalent  delay,  the  one  with  the  slow  feel  system  has  the  best 
pilot  rating.  Because  the  total  amount  of  delay  is  the  same  in  both  cases,  the  amount  of 
pure  delay  in  the  fast  system  is  the  largest.  The  rating  seems  to  be  determined  by  this 
pure  delay  only.  Smith  and  Sarrafian  therefore  question  the  maximum  delay  values  set 
in  the  military  standard  MIL-F-8785C,  because  it  doesn’t  make  this  distinction. 

However,  David.  Bimal  and  David  (1992)  also  found  that  feel  system  equivalent  delay 
differs  from  pure  transport  delay,  but  that  there  are  also  correlations.  They  thought  it  too 
early  to  discount  the  contribution  of  the  feel  system’s  equivalent  delay  from  the 
required  maximum  set  by  MIL-F-8785C. 

In  a  one-dimensional  tracking  task,  Hess  (1984)  showed  that  vehicle  dynamics  can  have 
a  dramatic  effect  upon  airplane-pilot  coupling  especially  when  time  delays  are  present. 
He  used  simple  gain,  first  and  second  order  dynamics.  Subjects  couldn’t  complete  the 
task  with  the  latter  dynamics  and  a  delay  of  357  ms.  At  a  delay  of  214  ms  performance 
has  deteriorated  somewhat,  but  not  dramatically.  What  did  change  was  the  ability  of  the 
closed-loop  pilot-vehicle  response  to  abrupt,  transient  inputs. 
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Figure  1 3  Example  of  a  Cooper-Harper  handling  qualities  rating  chart. 


3.5  Simulators 

Delays  have  always  been  an  issue  in  flight  simulators.  Not  only  do  they  deteriorate  the 
performance  of  the  pilot,  but  also  quite  often  lead  to  simulator  sickness  or  at  least  to 
various  unpleasant  feelings.  Research  in  simulators  often  focuses  on  delays  in  the 
computer  generated  graphics  and/or  the  motion  system,  which  is  not  quite  the  same  as 
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the  delays  to  and  from  the  controlled  vehicle  in  the  UAV  case,  but  has  comparable 
effects,  and  is  for  most  means  and  purposes  indistinguishable  by  the  operator  from 
control  delay  (Crane,  1983). 

Frank,  Casali  and  Wierwille  (1988)  examined  the  effects  of  both  visual  and  motion 
delays.  Both  delays  contribute  to  performance  degradation  and  uneasiness,  but  delays  in 
visuals  more  than  delays  in  motion.  Performance  deteriorates  in  proportion  to  amount  of 
delay;  both  components  contribute.  Uneasiness  reaches  a  peak  around  300  ms. 

On  the  other  hand,  Draper,  Viire,  Furness  and  Gawron  (2001)  did  not  find  an  effect  of 
extra  delay  on  sickness.  A  comparison  across  experiments  suggested  no  appreciable 
increase  in  simulator  sickness  with  increasing  extra  time  delays  (125  and  250  ms)  above 
the  nominal  value  (48  ms). 

Whiteley  and  Lusk  (1990)  examined  landing  performance  under  conditions  with  delay. 
Landing  performance  with  a  90  ms  delay  was  better  than  200  ms  delay.  In  a  sidestep 
maneuver  scenario  they  found  significant  differences  in  roll  and  heading  control  for  90, 
200  and  300  ms  delay,  but  no  significant  difference  in  crashes.  They  reported  more 
control  stick  activity  with  increasing  delay. 

Gawron,  Bailey,  Knotts  and  McMillan  (1989)  examined  the  effects  of  pure  transport 
delay  added  to  the  pitch  and  roll  flight  control  system  on  pilot  performance  using  the 
NASA  variable-stability  NT-33A  aircraft.  They  compared  the  results  with  those  of  a 
fixed-based  flight  simulator.  Generally,  they  found  that  the  effects  of  the  delays  were 
more  severe  for  a  simulated  F-16  (fighter)  airplane  than  a  C-141  (transporter)  airplane. 
For  the  F-16,  roll  errors  started  to  increase  after  about  40  ms,  increasing  almost  twice  as 
fast  in  the  simulator  than  in  the  real  airplane  adding  17%  and  1 1%  to  their  respective 
baseline  errors  for  each  100  ms  of  additional  delay.  Pitch  errors  started  to  increase  after 
about  30  ms  adding  17%  and  25%  to  their  respective  baseline  errors  for  each  100  ms  of 
additional  delay.  For  the  C-141,  roll  errors  started  to  increase  after  about  60  and  1 10  ms 
for  the  simulated  and  real  airplane,  respectively,  adding  13%  and  19%  to  their 
respective  baseline  errors  for  each  100  ms  of  additional  delay.  Pitch  errors  started  to 
increase  after  about  50  and  90  ms  for  the  simulated  and  real  airplane,  respectively, 
adding  17%  and  25%  to  their  respective  baseline  errors  for  each  100  ms  of  additional 
delay.  In  short:  for  slow  airplanes  longer  delays  are  tolerable  and  in  simulators  (without 
motion  feedback)  performance  starts  to  deteriorate  earlier  than  in  the  real  airplane  (with 
motion  feedback). 

Bailey,  Knotts,  Horowitz  and  McMillan  (1987)  measured  degradation  in  ground-based 
simulation  versus  in-flight  simulation  using  a  Cooper-Harper  rating  scale.  They  found  a 
one  C-H  rating  unit  decrease  for  100  ms  delay  in  ground-based  simulation  and  1.5  unit 
for  in-flight  simulation.  Degradation  starts  at  about  130  ms  of  added  delay  (with  a  100 
ms  baseline),  though  the  authors  mention  in  their  conclusions  a  number  of  50  ms  (+100 
ms  baseline)  as  an  acceptable  delay.  In  the  study,  pilots  developed  different  control 
strategies  to  handle  delays.  Some  pilots  used  low  gain  control;  others  used  high 
frequency  pulsing  type  input,  and  some  tightened  their  grip  on  the  controls. 

Miller  and  Riley  (1977)  examined  visual  delays  and  compared  moving  base  and  fixed- 
base  simulators.  They  found  that  ‘bad’  planes  (with  a  low  C-H  rating  of  6)  allow  for 
only  47  (baseline)  +  31  ms  (additive)  delay.  With  good  ones  (with  a  C-H  rating  of  3.5), 
a  47  +  3*31  =  150  ms  could  be  handled.  They  stated  that  in  the  case  of  bad  planes  no 
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added  delay  is  actually  possible.  The  effect  of  the  motion  base  was  significant  even  at  0 
delay.  For  a  basic  plane  the  effect  was  found  in  the  fixed  base  after  120  ms  delay,  and  in 
the  moving  base  after  240  ms.  The  numbers  for  a  good  plane  were  120-240  ms  and  360 
ms,  respectively.  At  high  workloads,  lower  delays  were  found  to  be  acceptable. 

The  DIS  (Distributed  Interactive  Simulation)  standard  specifies  a  maximum  end-to-end 
latency,  i.e.  time  from  onset  of  an  action  to  a  corresponding  change  in  the  simulator’s 
output,  of  less  than  300  ms  for  ‘loosely  coupled’  interactions  and  less  than  100  ms  for 
‘tightly  coupled’  interactions,  e.g.  formation  flying.  DIS  works  by  sending  various  data 
packages  over  a  simulation  network.  A  position  package  is  used  to  update  position, 
velocity  and  acceleration.  While  waiting  for  the  next  packages  to  arrive  DIS  uses  the 
entities’  ‘dead-reckoned’  position.  This  is  an  extrapolated  position  estimated  based  on  a 
simple  motion  calculation  using  known  velocity  and  acceleration  (the  latter  is  not 
obligatory).  The  dead-reckoning  process  is  meant  to  deal  with  both  low  update  rates  and 
latencies.  In  two  experiments,  De  Vries  (1999)  and  De  Vries  &  Kappe  (1999)  examined 
the  effect  of  delay  in  a  DIS-like  simulation  in  which  two  vehicle  controllers  had  to 
minimize  steering  errors  with  respect  to  their  own  vehicle  paths  and  were  required  to 
line-up  their  respective  vehicles.  The  experiments  simulated  both  internal  delays  (delays 
within  the  vehicle’s  own  simulator  loop)  and  external  delays  (transport  delays  between 
simulators).  In  both  experiments,  line-up  errors  depended  more  strongly  on  the  internal 
delay  than  on  the  external  delay.  The  contribution  of  the  internal  delay  to  the  line-up 
error  was  2-4  times  as  high  as  that  of  the  external  delays.  All  delays,  even  the  smallest 
delay  used  (50  ms),  contributed  to  a  clearly  measurable  increase  in  the  errors  (about 
15%  at  50  ms),  growing  linearly  with  the  amount  of  delay.  The  experiments  also 
showed  that  amount  of  control  movements,  and  thus  workload,  are  more  strongly 
influenced  by  delays  than  the  errors.  This  is  consistent  with  the  results  of  Cooper, 

Harris  and  Sharkey  (1975)  who  found  that  differences  in  steering  behaviour  may  occur 
even  without  alteration  of  the  performance. 

3.6  UAVs 

Research  on  UAVs  control  can  be  divided  into  two  categories:  vehicle  control  and 
sensor  control.  This  division  may  be  a  bit  artificial:  For  a  UAV  operator  both  vehicle 
and  sensor  controls  often  have  an  almost  indistinguishable  effect  on  the  resulting  sensor 
image  that  serves  as  the  operator’s  feedback.  Flying  in  a  straight  line  while  rotating  the 
camera  may  result  in  comparable  optic  flow  as  flying  a  curved  path  with  the  camera 
steady.  One  difference  in  practice  will  be  that  the  vehicle  will  respond  much  slower 
than  the  sensor. 

In  his  thesis  on  UAV  handling  qualities,  Thurling  (2002)  cites  the  RPV  Flying  Qualities 
Design  Criteria  (Prosser  &  Wiler,  1976).  “The  authors  expected  the  stability  and 
response  characteristics  of  an  RPV  to  be  different  from  a  piloted  aircraft.  With  the  pilot 
removed  from  the  proprioceptive  feedback  cues  in  the  cockpit,  traditional  values  for 
response  parameters  may  not  apply.”  In  Thurling’s  study,  with  the  NASA  twin-engine 
Utility  UAV  and  pilots  as  UAV  controllers,  189  ms  is  about  the  maximum  additional 
time  delay  that  can  be  allowed  for  an  uncompensated  system.  With  this  amount  of 
delay,  five  out  of  the  six  operators  involved  in  the  study  rated  the  UAV  as  level  3 
(Cooper- Harper  rating  7-9).  More  results  like  this  can  be  found.  Hall  et  al.  (2001),  for 
instance,  found  that  Cooper-Harper  ratings  of  operators  controlling  a  UAV  camera  are 
significantly  influenced  by  latencies  of  300  ms  and  higher. 


ONGERUBRICEERD 


time  to  locate  the  six  oil-rigs  (s) 


ONGERUBRICEERD  |  TNO  report  |  TNO-DV3  2005  A054 


31/50 


A  model  study  (with  a  fairly  simple  UCAV  model)  by  Dougherty  (2002)  shows  that 
unaided  laser  designation  by  the  controller  of  a  UCAV  leads  to  errors  for  maneuvering 
targets.  The  errors  depend  on  the  amount  of  latency,  and  the  speed  and  maneuvering 
type  (constant  velocity,  accelerating)  of  the  target.  UC  AV-GCS  delays  of  300  ms  lead 
to  designation  misses  of  around  70  ft. 

Apparently,  200-300  ms  is  about  acceptable,  but  not  more.  Van  Erp  &  Kappe  (1998) 
explored  much  longer  delays.  They  had  subjects  control  an  image  sensor  using  a  radar 
display  that  showed  the  sensor’s  footprint.  The  authors  found  that  search  time  increased 
by  250%  for  1  s  delay,  by  350%  for  2  s  and  by  400%  for  4  s  of  delay.  These  data  are  an 
average  over  the  (low)  update  rates  used  in  the  experiment  (0.5,  2  and  4  Hz)  compared 
to  the  control  condition  with  30  Hz.  The  results  also  show  that  the  distances  ‘traveled’ 
by  the  camera’s  footprint  did  not  increase  as  fast.  Therefore,  a  large  part  of  the  increase 
in  search  time  results  from  slower  control  movements,  whereas  a  smaller  part  results 
from  less  efficient  control  movements  (overshoots). 

A  number  of  experiments  involved  a  situation  in  which  both  the  UAV  and  its  sensor 
were  controlled  by  a  UAV  operator,  using  head-coupled  control  for  the  sensor:  A  head- 
tracker  translates  head  movements  in  control  instructions  for  the  UAV  sensor  the 
images  of  which  are  presented  on  a  UAV.  Grunwald,  Kohn  &  Merhav  (1991)  using  a 
lag  of  500  ms  in  the  head-coupled  loop  in  a  UAV  simulation  experiment  found  only  a 
small  influence  (4%  higher  error  scores)  on  platform  control  but  a  large  impact  on  head 
movements.  Other  researchers  (De  Vries  &  Padmos,  1997)  found  that  adding  50  ms 
delay  to  the  UAV  imagery  already  delayed  by  a  simulator-determined  delay  of  70-150 
ms  increased  UAV  control  errors  by  about  20%. 

In  an  experiment  where  a  UAV  operator  using  a  head-coupled  camera  performed  a 
patrolling  task  (Van  Erp  &  Van  Breda,  2001;  Van  Erp  &  Van  der  Dobbelsteen,  1998a; 
Van  Erp  &  Van  der  Dobbelsteen,  1998b),  most  tasks  were  negatively  influenced 
beyond  500  ms.  With  a  one  second  delay,  one  of  the  search  tasks  (finding  the  location 
of  six  oil  rigs)  took  more  than  twice  the  amount  of  time  to  finish  and  it  took  more  than 
five  times  with  a  delay  of  4  s  (see  Figure  14,  left-hand  panel).  Not  all  performance 
indicators  were  found  to  be  as  sensitive  for  transmission  delays.  For  one  indicator 
measuring  the  standard  deviation  of  the  UAV  camera  pitch  and  hence  the  amount  of  its 
activity,  only  a  delay  of  4  s  sufficed  to  yield  a  significant  decrease  in  performance  (see 
Figure  14,  right-hand  panel). 


transmission  delay  (s) 


transmission  delay  (s) 


transmission  delay  (s) 


Figure  14  Data  from  a  UAV  patrol  experiment  by  Van  Erp  and  Van  der  Dobbelsteen  (1998a).  Search 
performance  is  indicated  by  three  different  indicators,  from  left  to  right:  The  time  it  took  to 
locate  a  configuration  of  6  oil  rigs  (the  configuration  was  much  larger  than  the  field  of  view  of 
the  camera),  the  time  it  took  to  react  to  a  penetration  of  the  oil  rig  area  by  a  ship  and  the 
standard  deviation  of  the  camera’s  pitch,  a  measure  that  describes  the  amount  of  activity  of  the 
camera. 
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4  Solutions 


4.1  Adaptation  &  training 

Vehicle  controllers  can  often  learn  to  adapt  to  the  behavior  of  their  vehicles.  By  means 
of  experimentation,  training,  and  extensive  exposure  to  the  system  its  characteristics  can 
be  more  or  less  internalized  and  the  vehicle  can  be  partly  controlled  in  an  open-loop 
way  (feed-forward).  The  control  lead  generated  by  the  controller  comes  at  the  cost  of  an 
increased  workload  and  it  has  its  limits.  Nevertheless,  it  is  almost  always  fruitful  to 
train  extensively. 

Ricard  (1995)  tested  the  number  of  trials  necessary  to  reach  criterion  level  performance 
in  controlling  pitch  and  roll  of  a  simple  airplane  model.  It  appears  that  for  small  delays 
(less  than  200  ms)  the  same  performance  can  be  achieved  as  for  zero  delay  with 
sufficient  training.  Ricard  found  that  various  compensation  schemes  could  accelerate 
the  training  time  considerably  (see  Figure  15). 

Other  researchers  (Foulkes  &  Miall,  2000)  studied  the  effects  of  training  as  well. 

Though  they  report  significant  improvement  in  errors  (about  20%  less  errors  after 
training  for  a  300  ms  delay  and  15%  for  a  200  ms  delay)  they  do  not  confirm  Ricard’ s 
finding  that  the  delays  can  be  compensated  totally.  In  fact,  even  after  prolonged  training 
the  subjects  in  the  200  ms  delay  group  still  had  a  60%  higher  error  score  than  the  no¬ 
delay  group. 


ADDED  DELAY  IN  MILLISECONDS 

Figure  1 5  Number  of  trials  required  to  reach  to  criterion  performance  for  three  different  systems  (a 
delayed  system  and  two  delayed  systems  with  added  filters)  as  a  function  of  added  delay 
(Ricard,  1995).  In  each  case,  the  same  level  of  performance  is  achieved,  but  the  amount  of 
training  (and  probably  effort)  involved  varies  with  the  delay. 
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4.2  Filtering  and  prediction 

Quite  often  filtering  of  the  control  signals  leads  to  an  improvement  of  operator 
performance.  Some  filters,  like  a  low-pass  filter,  prevent  the  occurrence  of  Pilot- 
Induced  Oscilations  (PIOs)  because  they  cause  an  initial  decrease  of  the  acceleration  of 
the  response  on  a  given  error  signal:  they  lower  peak  gain. 

A  good  example  of  a  delay-compensating  filter  is  the  so-called  lead-lag  filter.  An 
example  of  such  a  filter  (Van  de  Vegte,  1990)  and  its  Bode  plot  is  given  in  Figure  16. 
Its  transfer  function  is: 


/(,>=  »+i , 

tit2s  +  (tl  +t2  +tn)s  +  \ 


(1) 


with  tx  =  RlCl ,  t2  =  R2C2 .  and  tX2  =  /?,C12  .  The  origin  of  the  name  of  the  lead-lag 
filter  is  clear  from  the  phase  diagram:  Higher  frequencies  get  a  phase  lead,  whereas  the 
lower  frequencies  get  a  phase  lag.  The  amplitude  diagram  shows  attenuation  in  the 
midrange  while  low  and  high  frequencies  are  not  attenuated  (notch  filter).  With  an 
optimal  choice  of  parameters,  this  behavior  can  deal  with  the  delay  problem.  The  phase 
lead  works  as  a  kind  of  predictor,  projecting  part  of  the  control  response  into  the  future, 
which  to  some  extent,  compensates  for  the  delay. 
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Figure  16  Electronic  circuit  example  of  a  lead-lag  network  and  its  Bode  plot.  The  members  of  the  lead- 
lag  family  look  similar  to  this  example,  but  the  specific  location  of  the  amplitude  notch  and 
their  depths  as  well  as  those  of  the  phase  lead  and  lag  lobes  depend  on  the  specific  values  of  the 
parameters  (in  this  case  Ri,  R2,  C\,  and  C2). 


How  does  such  a  filter  improve  the  operator’s  performance?  Using  the  simulation  set¬ 
up  shown  in  Figure  17,  we  compare  the  simple  operator/vehicle/delay  system  we 
encountered  earlier  (Figure  12,  reflected  in  the  upper  part  of  Figure  17)  with  a  system  in 
which  the  operator’s  control  output  is  filtered  by  a  lead-lag  filter.  In  Figure  18  the 
response  of  both  systems  with  a  varying  amount  of  delay  to  a  step  input  (for  instance 
the  requirement  to  suddenly  change  flight  levels)  is  shown.  The  response  to  a  sine  wave 
input  with  varying  frequencies  is  given  in  Figure  19.  Clearly,  the  filtered  system  is  far 
more  robust  and  less  prone  to  PIOs.  The  first  panel  of  Figure  18  shows  that  it  is 
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important  to  tune  the  filter  to  the  amount  of  delay  and  the  typical  frequencies  in  the 
control  regime.  An  incorrect  choice  may  result  in  a  response  that  is  less  adequate  than 
the  uncompensated  system. 


Figure  17  A  Matl^ab  Simulink  model  used  to  compare  a  simple  delayed  system,  in  this  case  an  integrator 
with  a  delay  on  the  feedback  line,  with  the  same  system  in  which  the  control  output  of  the 
operator  (modeled  by  the  comparator)  is  filtered  with  a  lead-lag  filter. 
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Figure  18  Output  of  the  model  of  Figure  17  for  delays  0.5,  0.9  and  1.1s,  respectively.  The  other  values 
used  in  the  model  were:  gain  =  1.5,  ti  =  5,  t2  =  1  and  ti2  =  10  s.  The  input  function  (a  step 
function)  is  shown  in  red;  the  result  with  the  unfiltered  operator  response  is  shown  in  blue, 
while  the  filtered  response  is  shown  in  green.  Note  that  the  vertical  scale  is  different  in  each 
graph. 
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Figure  19  Output  of  the  model  of  Figure  17  with  a  sine  wave  as  input  functions.  Shown  are  results  with 
input  sine  frequencies  of  1, 0.5,  and  0.25  Hz,  respectively.  The  other  values  used  in  the  model 
were:  gain  =  1.5,  ti  =  5,  t2  =  1  and  tj2  =10  s.  The  input  function  (a  sine  function)  is  shown  in 
red,  the  unfiltered  response  is  shown  in  blue,  and  the  filtered  response  is  shown  in  green.  Note 
that  the  vertical  scale  is  different  in  each  graph  and  that  the  horizontal  scale  of  the  first  graph 
differs  from  the  others. 

Of  course,  with  a  real  human  operator  in  the  loop,  results  will  be  different.  The  reason  is 
that  human  operators  themselves  are  more  complex  than  the  comparator  in  Figure  17, 
and  may  be  able  to  generate  some  lead,  just  as  the  lead-lag  filter  does. 
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Various  filters  have  been  tried  and  are  described  in  the  literature  and  most  of  them  are 
able  to  compensate  for  considerable  delays. 

In  the  research  mentioned  in  Section  4.1,  Ricard  (1995)  found  that  lead-lag  filtering  not 
only  may  improve  performance,  but  that  it  also  decreases  skill  acquisition  time 
considerably. 

Shafer,  Smith,  Stewart  and  Bailey  (1984)  tested  two  PIO  filters  (scarcely  described  in 
the  article)  that  are  used  in  the  space  shuttle.  Using  two  NASA  test  aircraft  and  trying 
various  parameters  of  their  filters,  they  found  that  they  could  lower  the  Cooper-Harper 
rating  (see  Figure  13)  with  3  points  even  with  an  added  delay  of  165  ms. 

Crane  (1983)  examined  various  systems  with  a  delay  of  108  ms  and  found  that  all  five 
test  pilots  that  took  part  in  his  simulator  experiment  benefited  from  delay  compensation, 
some  pilots  even  performed  better  with  the  delay  compensated  system  with  added  delay 
than  in  the  uncompensated  baseline  system  without  added  delay. 

Sobiski  and  Cardullo  (1987)  claim  that  lead-lag  filtering  as  performed  by  Ricard  and 
Harris  (1980)  represents  suboptimal  compensation.  Apart  from  a  lead-lag  filter  they  test 
a  predictive  method  in  which  the  state  transition  matrix  is  applied  in  the  feedback  loop 
to  compensate  for  time  delays.  In  an  experiment  in  which  participants  were  required  to 
stabilize  an  attitude  indicator  (artificial  horizon  display),  they  did  not  find  positive 
effects  of  lead-lag  filtering,  but  their  own  method  appears  to  compensate  for  delays  of 
up  to  800  ms.  Their  lead-lag  filter  differs  from  the  one  in  Equation  1,  though.  They  used 
filters  of  the  form: 


,f(s)  = 


TjS  +  l’ 


(2) 


which,  according  to  Feedback  Control  Systems  textbook  author  Van  de  Vegte  (1990),  is 
called  a  phase  lag  filter;  apparently  definitions  vary.  The  filter  has  a  Bode  plot  that 
differs  considerably  from  that  of  the  lead-lag  filter  of  Equation  1.  An  additional  note: 
The  Sobiski  and  Cardullo  study  is  a  study  of  the  compensation  of  the  delayed  visuals  of 
a  simulator.  Delays  were  only  present  in  the  feedback  part  of  the  loop  and  not  in  the 
control  part  as  would  be  the  case  for  a  UAV.  The  prediction  used  in  their  method  is  a 
prediction  of  the  future  state  of  the  simulator  visuals  corresponding,  of  course,  to  a 
future  state  of  the  simulated  vehicle,  which  would  be  hard  to  do  for  the  real  imagery  of 
a  UAV  (however,  see  next  section). 

The  results  of  Sobiski  and  Cardullo  are  confirmed  in  a  study  by  Cardullo  and  George 
(1993).  They  compare  the  delay  compensation  offered  by  a  lead-lag  filter  (again  of  the 
type  of  Equation  2,  not  1)  with  that  of  a  McFarland  predictor  (McFarland,  1988)  and  the 
Sobiski  and  Cardullo  (1987)  predictor.  This  study  was  a  model  study  and  did  not  use 
human  operators  as  Sobiski  and  Cardullo  did,  but  modeled  them  by  a  transfer  function. 
The  results  were  comparable:  The  predictor  schemes  are  able  to  compensate  fully  for 
delays  up  to  400  ms,  and  at  800  ms  perform  only  slightly  worse  than  the  non-delayed 
system.  In  a  recent  paper  (Guo,  Cardullo,  Houck,  Kelly  &  Wolters,  2004),  the 
predictive  methods  were  elaborated  using  either  a  Kalman  estimator  or  a  state  space 
predictive  filter  and  considerably  improved,  as  the  described  model  study  seems  to 
indicate. 


ONGERUBRICEERD 


ONGERUBRICEERD  |  TNO  report  |  TNO-DV3  2005  A054 


38/50 


In  research  specifically  aimed  at  UAVs  (Thurling,  2002),  a  system  compensated  with  a 
model-based  prediction  scheme,  389  ms  was  the  maximum  useable  delay.  Thurling 
used  a  model-based  prediction,  because  simulations  showed  it  to  be  better  than  a  lead- 
lag  filtering  scheme.  The  compensated  system,  the  NASA  twin-engine  Utility  UAV, 
had  a  good  Cooper-Harper  rating:  five  out  of  six  operators  rated  the  UAV  as  a  level  2 
system  (flyable  with  some  difficulties,  Cooper-Harper  rating  4-7),  averaging  a  C-H 
rating  of  5.3.  This  is  much  better  than  the  uncompensated  system  with  a  delay  of  189 
ms  which  was  rated  as  a  level  3  plane.  Five  out  of  six  operators  rated  the  UAV  as  level 
3  (Cooper- Harper  rating  7-9).  Additionally,  pilots  stated  that  the  compensated  system 
did  not  show  PIO  tendencies  and  workload  was  much  lower  with  the  compensation  than 
without.  Improvements  in  objective  measurements  were  not  so  univocally  positive.  One 
of  the  two  error  measurement  variables  used  in  the  experiment  was  improved  with 
prediction,  but  the  other  was  not.  Thurling  hypothesizes  that  shortcomings  in  the 
prediction  scheme  were  most  likely  due  to  inadequacies  in  the  linear  model  used  to 
generate  the  predictive  display  or  unmodeled  disturbances  such  as  wind  gusts  and 
turbulence. 

More  evidence  on  the  positive  effects  of  prediction  can  be  found  in  two  studies  on  the 
networked  simulation  protocol  DIS  (De  Vries,  1999;  De  Vries  &  Kappe,  1999).  In  these 
studies,  it  was  found  that  internal  delays  (delays  within  a  simulator)  contributed  two  to 
four  times  as  much  to  formation  flying  errors  as  external  delays  (delays  between  two 
simulators).  The  difference  between  the  two  types  of  delay  in  this  experiment  was  that 
the  internal  information,  though  being  processed  at  a  high  update  rate,  was  not 
compensated  for  the  delay  whereas  the  external  data  stream,  sent  at  a  low  update  rate, 
was.  The  position  information  of  the  external  vehicle  was  extrapolated  using  a  second 
order  predictive  motion  scheme  (constant  acceleration).  Though  prediction  clearly 
helped  lowering  average  errors,  it  did  increase  the  frequency  of  the  error  patterns, 
indicating  either  a  higher  sensitivity  to  noise  or  a  higher  workload. 

In  a  study  aimed  at  a  UAV  sensor  operator.  Van  Erp  and  Kappe  (1998)  reported  very 
good  results  of  camera  movement  prediction.  Search  times  were  much  lower  (by 
roughly  50%)  with  prediction  and  increased  less  strongly  with  increasing  delays  (see 
Figure  20). 
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Figure  20  Data  from  Van  Hrp  and  Kappe  (1998).  Search  time  and  search  efficiency  (as  indicated  by  the 
distance  the  camera  footprint  traveled  to  reach  its  goal)  as  a  function  of  transmission  delay. 
Note  that  these  results  are  an  average  of  several  low  update  conditions.  The  control  condition 
has  both  a  high  update  rate  and  a  zero  added  transmission  delay. 


4.3  Augmented  displays 

Augmented  displays  techniques  are  closely  related  to  prediction  techniques.  The  basic 
idea  is  that  the  delayed  visuals  are  embedded  in  imagery  that  is  not  delayed,  i.e.  the 
display  shows  the  context  of  the  visuals  as  if  it  was  taken  with  a  sensor  that  responds 
immediately  to  the  operator’s  control  instructions. 

In  this  way,  not  only  the  operator’s  situational  awareness  is  improved  by  placing  the 
image  in  its  geographical  context,  but  the  PIO  tendency  of  the  operator-UAV  system  is 
greatly  reduced  as  well.  The  latter  is  possible,  because  most  of  the  operator’s  control 
task  is  now  moved  to  the  non-delayed  area.  Examples  of  augmented  displays  can  be 
seen  in  Figure  21  and  Figure  22,  imagery  augmented  with  2D  information  in  a 
perspective  presentation,  and  with  mixed  2D/3D  information,  respectively. 
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The  information  with  which  the  UAV  imagery  is  augmented  has  to  be  available  of 
course.  The  required  extra  information  ranges  from  almost  none  Figure  21a,  to  area 
maps  Figure  21c,  and  to  complete  3D  models  of  the  theatre  of  operations  (Figure 
22b,d).  Nowadays,  even  the  latter  information  is  relatively  easy  to  obtain. 

Another  requirement  that  is  less  easily  obtained  is  a  correct  prediction  of  the  sensor’s 
future  state  after  the  current  operator’s  control  instructions  have  arrived.  This  means  we 
need  a  very  accurate  model  of  the  vehicle  and  its  sensor,  as  well  as  of  their  environment 
(wind,  turbulence). 


c  d 

Figure  2 1  Screen  shots  from  the  TNO  Human  Factors  UAV  demonstrator.  In  the  centre  of  all  panels  the 
(simulated)  camera  image  of  a  UAV  sensor  can  be  seen,  a:  The  camera  image  is  surrounded  by 
a  grid,  generated  at  the  GCS  which  predicts  the  effects  of  (delayed)  camera  control  movements. 
b:  The  grid  can  overlay  the  camera  image  and  can  in  this  way  convey  information  on  the  zoom 
state  of  the  camera,  c:  A  perspectively  projected  map  used  as  background,  d:  All  kinds  of  data 
can  be  used  to  augment  camera  images.  In  this  case  a  simulator  database  is  shown  in  wireframe 
mode.  Again,  this  can  be  used  to  overlay  the  camera  image  as  well.  In  this  way  underground 
pipelines  can  be  indicated,  for  instance. 


In  an  experiment  performed  at  TNO  by  Veltman  and  Oving  (2003),  participants  acting 
as  the  sensor  operator  of  a  UAV  inspected  roads  and  edges  of  wood  using  the  sensor 
with  and  without  the  aid  of  a  3D  map.  A  2D  map  with  the  same  information  as  the  3D 
map  was  available  in  all  conditions  (see  Figure  22).  With  the  3D  map,  the  participants 
were  able  to  inspect  larger  areas,  especially  when  the  task  became  more  difficult  due  to 
time  delays  and  low  update  rates,  were  better  able  to  perform  an  additional  task,  and 
reported  lower  workload  compared  to  the  condition  without  the  3D  map. 
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In  our  informal  experience  with  the  augmented  display  of  Figure  22,  the  sensor  can  be 
easily  operated  with  one  to  several  seconds  of  delay. 

As  an  interesting  side  note:  predictive  augmented  displays  have  been  intensively  tested 
for  application  in  tele-robotic  applications  in  space  for  some  time  now.  Quite  often 
delays  can  run  in  the  several  seconds,  and  great  improvements  have  been  reported  for 
using  augmented  displays  (Lane  et  al.,  2002;  Lane,  Carignan  &  Akin,  2000;  Lane, 
Carignan  &  Akin,  2001). 


Figure  22  Screen  shots  from  the  experiment  of  Veltman  and  Oving  (2003).  a:  a  2D  situation  display 

showing  the  camera  footprint,  i.e.  the  part  of  the  world  visible  on  the  camera  display.  Overlays 
in  two  colors  are  present:  The  yellow  color  indicates  the  predicted  camera  footprint;  the  orange 
color  indicates  the  position  of  the  footprint  corresponding  to  the  current  camera  image.  In  panel 
a  and  b  these  elements  overlap  and  can  not  be  distinguished  very  well;  in  c  and  d  the  camera  is 
in  motion  and  the  colored  elements  can  be  clearly  distinguished,  b:  a  3D  situation  display  with 
the  camera  image  as  an  insert  at  the  bottom  of  the  display.  The  situation  corresponds  to  the 
situation  depicted  in  a.  c:  the  camera  is  zoomed  in  more  (footprint  gets  smaller)  and  moves. 

The  boxes  indicate  the  part  of  the  3D  database  that  is  (orange  box)  and  will  be  (yellow  box) 
displayed.  Due  to  transmission  delays,  predicted  and  current  footprint  do  not  coincide  anymore. 
d:  The  3D  version  of  the  situation  depicted  in  panel  c. 
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4.4  Image  deflection 

It  is  an  option  to  show  sensor  imagery  on  an  HMD  or  on  a  simulator  projection  dome 
and  to  use  head-tracking  to  control  the  orientation  of  the  sensor  instead  of  manually 
controlling  the  sensor.  The  advantage  would  be  that  the  situational  awareness  of  the 
sensor  operator  would  be  improved  (where-you-look-is-what-you-get),  although  this 
purported  advantage  didn’t  show  up  in  a  previous  study  (De  Vries,  2001). 

The  problem  is,  data  link  latency  does  not  only  cause  head-coupled  images  to  be 
displayed  too  late,  but  they  are  also  displayed  in  the  wrong  position,  depending  on  the 
speed  of  the  head  movements.  If  the  latency  is  given  by  dt  and  the  head  motion  is  v 
deg/s  then  the  misplacement  of  the  image  is  v  dt  deg.  Since  head  motions  can  be 
hundreds  deg/s  the  images  may  be  off  by  tens  of  degrees.  Because  head  motion  varies 
considerably,  the  misplacement  of  the  image  in  space  does  vary  considerably  as  well, 
yielding  a  very  unstable  localization  of  the  image. 

Various  researchers  (So  &  Griffin,  1993;  So,  1997;  So  &  Griffin,  1997;  Van  Erp  &  Van 
der  Dobbelsteen,  1998a;  Van  Erp  &  Van  der  Dobbelsteen,  1998b)  have  conceived  and 
tested  a  comparable  solution  to  this  problem.  In  (So  &  Griffin,  1993)  several  other 
researchers  are  cited  as  well.  There  are  even  some  patented  solutions  (Allen, 

Portoghese,  Hebb  &  Breglia,  1984;  Lee  &  McCreary,  1984)  that  all  apply  more  or  less 
the  same  principle. 

The  solution  requires  knowledge  of  the  amount  of  latency,  which  is  usually  easily  to 
obtain  from  signal  time  stamps.  With  knowledge  of  the  latency,  it  is  easy  to  present  the 
image  at  the  correct  position.  This  means  that  the  image  is  not  always  in  front  of  the 
observers.  Though  this  means  that  the  image  frame  is  lagging  behind  the  head 
movements,  its  contents  are  displayed  at  the  correct  position  (at  least  when  these 
contents  are  stationary).  All  researchers  found  this  method  to  be  very  successful.  So  and 
Griffin  (1993)  reported  that  subjective  difficulty  rating  were  largely  unaffected  by  lags 
of  up  to  400  ms  if  deflection  and  prediction  was  used.  So  (1997)  found  that  image 
deflection,  combined  with  phase  lead  filters,  produced  a  tracking  performance 
unaffected  by  lag  that  amounted  in  his  experiment  to  140  ms. 

Van  Erp  and  Van  der  Dobbelsteen  (1998b)  found  a  considerable  performance  increase 
due  to  image  deflection  (or  “delay  handling”  as  they  called  the  method).  The 
improvements  of  delay  compensation  on  tasks  performed  with  a  set  of  delays  ranging 
from  0  to  4000  ms  averaged  1 5%  for  one  particular  task  and  40%  for  another.  In  a 
follow-up  experiment  (Van  Erp  &  Van  der  Dobbelsteen,  1998a),  they  did  not  find  much 
improvement  using  this  compensation  technique  if  the  images  were  zoomed-in. 

The  deflection  technique  could  be  combined  with  the  augmented  display  of  Figure  22b. 
The  small  sensor  window  showing  the  delayed  image  could  be  shown  at  a  position 
corresponding  to  that  of  the  sensor  at  the  time  the  image  was  actually  made. 
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5  Conclusions 


Roundtrip  delays  in  a  UAV-GCS  configuration  range  from  a  minimum  of  about  100  ms 
in  simple  line-of-sight  situations  to  more  than  1600  ms  while  using  geostationary 
satellites  and  taking  other  sources  of  delay  into  account.  This  is  a  conservative 
estimation.  Much  larger  delays  are  quite  possible. 

Both  basic  and  applied  research  experiments  generally  point  to  a  delay  of  100  ms  as  an 
amount  that  has  a  measurable  impact  on  human  performance.  Aviation-related  literature 
and  regulations  indicate  that  a  delay  of  250-300  ms  may  be  regarded  as  the  maximum 
value  from  a  flying  handling  qualities  viewpoint. 

In  the  past,  the  RNLAF  has  indicated  to  be  interested  in  UA Vs  that  have  a  strategic 
operating  range  of  more  than  1000  km.  This  means  that  line-of-sight  datalinks  are  out  of 
the  question  and  geostationary  satellites  will  be  the  preferred  type  of  connection,  until 
the  moment  arrives  that  a  flexible  network  of  low-flying  relay  stations  is  in  operation 
and  available  for  RNLAF  operations. 

Given  the  preferred  datalink  and  its  corresponding  delays,  UA  V  handling  qualities  will 
suffer  considerably,  to  the  point  of  becoming  almost  impossible  to  handle  directly. 

Using  techniques  such  as  filtering  and  predictive  displays,  handling  qualities  can  be 
made  acceptable  for  delays  of  up  to  400  ms,  though  even  extreme  values  of  800  ms  are 
claimed. 

However,  the  delays  caused  by  geostationary  satellite  connections  will  be  so  high  that 
even  with  these  techniques  direct  control  will  be  extremely  difficult. 

A  distinction  between  the  critical,  high-gain  task  of  vehicle  control  and  the  less  critical, 
lower  gain  task  of  sensor  control  should  be  made.  Whereas  vehicle  control  with  all 
possible  augmentation  is  probably  not  acceptable  beyond  400  ms  delay,  augmented 
sensor  control  may  be  quite  acceptable  well  within  the  seconds  range. 
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7  Acronyms 


AOB 

Advanced  Operations  Base 

FOB 

Forward  Operations  Base 

GCS 

Ground  Control  Station 

GDT 

Ground  Data  Terminal 

GPS 

Global  Positioning  System 

HALE 

High  Altitude  Long  Endurance 

HF 

High  Frequency 

HQ 

Headquarter 

LEO 

Low-Earth  Orbit 

MALE 

Medium  Altitude  Long  Endurance 

MOB 

Main  Operations  Base 

NSWC 

Navy  Surface  Warfare  Center 

PIO 

Pilot  Induced  Oscillation 

RF 

Radio  Frequency 

TCS 

Tactical  Control  Station 

UAV 

Unmanned  Aerial  Vehicle 

UCAV 

Unmanned  Combat  Aerial  Vehicle 

UHF 

Ultra  High  Frequency 

VHF 

Very  High  Frequency 
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